Skip to main content

Bedrock Inform Integration

Integrate your Amazon Bedrock model invocation logs with the nOps platform to unlock unprecedented visibility into your AI spending. Unlike traditional cost reporting that only provides daily or monthly aggregates, this integration delivers second-by-second granularity for every single model invocation, complete with request metadata that enables precise cost allocation through custom tagging.

This changes everything - instead of being stuck with high-level cost summaries, you can now track, analyze, and allocate every individual API call to specific teams, projects, or business units.

What Do You Get with Bedrock Integration in Inform?​

With the Bedrock integration in Inform, you get:

  • Detailed Cost Analysis: View comprehensive breakdowns of your Bedrock expenses, including costs by model, token usage, request metadata, latency, and more - every API call becomes a line item you can analyze and allocate.
  • Usage Insights: Track and analyze your AI model usage patterns to optimize resource allocation and spending.
  • Budget Management: Set and monitor budgets for your Bedrock services with alerts for approaching thresholds.
  • Multi-Account Support: Monitor multiple AWS accounts with Bedrock usage from a single dashboard for better cost management across projects.
  • Automated Data Sync: Regular updates of your usage data ensure you always have the latest cost information.
  • Custom Reporting: Generate detailed reports on your Bedrock usage and costs for better financial planning - slice and dice by any request metadata tag you've defined.

Benefits of Integration​

  • Revolutionary Granularity
    Break free from monthly cost summaries. Track every single Bedrock API call with second-level precision - see exactly when each model was invoked, how long it took, and what it cost.

  • True Cost Allocation
    Finally allocate AI costs accurately. Request metadata becomes cost allocation tags, letting you attribute every dollar spent to specific teams, projects, customers, or business units.

  • Individual Call Analysis
    Stop guessing about AI usage patterns. Analyze token consumption, latency, and costs at the individual invocation level to identify optimization opportunities you've never seen before.

  • Real-Time Cost Control
    Monitor AI spending as it happens. Get immediate visibility into cost spikes and usage patterns instead of waiting for monthly bills.

  • Unprecedented Transparency
    Transform AI from a black box expense into a fully visible, manageable cost center. Every model, every call, every token - completely tracked and allocated.

FAQs​

Expand FAQs

1. How does the Bedrock integration with nOps work?​

This integration represents a breakthrough in AI cost visibility. While traditional cost reporting gives you daily and hourly aggregates like "you spent $1,000 on Bedrock today" our integration captures every single model invocation in real-time through Bedrock's Model Invocation Logs.

Here's what makes it revolutionary:

  • Second-level granularity: See exactly when each API call happened, not just daily/monthly totals
  • Request metadata preservation: Every custom tag and metadata you include in your API calls becomes a cost allocation dimension
  • Individual call tracking: Track token usage, latency, model type, and costs for every single invocation

Instead of wondering "why did our AI costs spike?" you can pinpoint exactly which team, project, or application drove the increase - down to the specific API call.

2. What permissions does the integration require?​

The integration requires access to CloudWatch Logs where Bedrock Model Invocation Logs are stored. During the nOps onboarding process, an inline NopsIntegrationPolicy is typically attached to your Nops-Integration-* IAM Role. You'll need to ensure proper permissions are configured for log access.

3. Can I use existing AWS integrations?​

Yes, the Bedrock integration works with your existing nOps AWS account integration. However, you'll need to enable Model Invocation Logs in Bedrock and ensure the necessary IAM permissions are configured for nOps to access the log data.

4. How long does it take for data to appear in nOps after setup?​

It may take up to 24 hours for the initial data synchronization to complete. After this period, your Bedrock usage data should be visible in the Cost Analysis tool.

5. What should I do if my usage data doesn't appear after 24 hours?​

If the data is not visible after 24 hours, check the following:

  • Verify that Model Invocation Logs are enabled in Bedrock
  • Confirm that the log group ARN is correctly configured
  • Ensure that GetLogEvents:* is not explicitly denied in your IAM policies
  • Check that there is actual Bedrock usage activity in your AWS account

If the issue persists, contact nOps support for assistance.

6. Can I monitor multiple AWS accounts with Bedrock?​

Yes, you can monitor multiple AWS accounts by setting up separate integrations for each account. Each integration requires its own AWS account connection and proper IAM permissions. This allows you to track costs across different projects, departments, or environments.

7. What Bedrock costs are tracked in the integration?​

The integration tracks all costs associated with your Bedrock usage, including:

  • Model invocation costs (Claude, Titan, Jurassic, etc.)
  • Input and output token usage
  • Custom model fine-tuning costs
  • Knowledge base usage
  • Any other billable Bedrock services

8. How do I enable Request Metadata for cost allocation?​

Request Metadata is the secret sauce for cost allocation. To enable it, include custom metadata in your Bedrock API calls:

Python SDK Example:

import boto3
import json

bedrock = boto3.client('bedrock-runtime')

response = bedrock.invoke_model(
    modelId='anthropic.claude-3-sonnet-20240229-v1:0',
    body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "messages": [{"role": "user", "content": "Hello!"}],
        "max_tokens": 100
    }),
    # This metadata becomes cost allocation tags!
    requestMetadata={
        'team': 'engineering',
        'project': 'chatbot-v2',
        'environment': 'production',
        'customer_id': '12345'
    }
)

Every key-value pair in requestMetadata becomes a cost allocation dimension in nOps. You can then slice your costs by team, project, customer, or any custom tag you define.